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Abstract 

We study the conditional sojourn time distributions of processor sharing (PS), fore- 
ground background processor sharing (FBPS) and shortest remaining processing time first 
(SRPT) scheduling disciplines on an event where the job size of a customer arriving in 
stationarity is smaller than exactly fc > out of the preceding m > k arrivals. Then, 
conditioning on the preceding event, the sojourn time distribution of this newly arriving 
customer behaves asymptotically the same as if the customer were served in isolation with 
a server of rate {1 — p)/{k + 1) for PS/FBPS, and (1 — p) for SRPT, respectively, where 
p is the traffic intensity. Hence, the introduced notion of conditional limits allows us to 
distinguish the asymptotic performance of the studied schedulers by showing that SRPT 
exhibits considerably better asymptotic behavior for relatively smaller jobs than PS/FBPS. 

Inspired by the preceding results, we propose an approximation to the SRPT discipline 
based on a novel adaptive job grouping mechanism that uses relative size comparison of 
a newly arriving job to the preceding m arrivals. Specifically, if the newly arriving job is 
smaller than k and larger than m — /c of the previous m jobs, it is routed into class k. 
Then, the classes of smaller jobs are served with higher priorities using the static priority 
scheduling. The good performance of this mechanism, even for a small number of classes 
m + 1, is demonstrated using the asymptotic queueing analysis under the heavy-tailed job 
requirements. We also discuss refinements of the comparison grouping mechanism that 
improve the accuracy of job classification at the expense of a small additional complexity. 

Keywords: Comparison scheduling, scalability, fairness, adaptive thresholds, M/G/1 
queue, processor sharing, shortest remaining processing time first, foreground background 
processor sharing, asymptotic analysis, heavy tails, medium size jobs 
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Introduction 



It has been widely recognized that heavy-tail ed distributions are suitabl e for ni o deling job 
sizes in information service networks, e.g., see Jelenkovic and Momcilovic ( 2003al lbl) and the 



references therein. For heavy-tailed distributions, large jobs appear much more frequently than 
for the light-tailed ones, which imposes very different constraints in terms of optimizing the 
scheduling process as compared to the light-tailed scenarios. In particular, schedulers that may 
assign the server exclusively to a very large job, e.g., first come first serve (FIF O) discipline, 
can ca use very large delays and, in general, suboptimal performance, as shown bv I Anantharam 
(|l999l ). 

Hence, most of the practical schedulers utilize either the processor sharing (PS) and fore- 
ground background processor sharing (FBPS) disciplines because of their inherent fairness, or 
the shortest remaining processing time first (SRPT) discipline because of its known optimality 
under quite general conditions. In particular, it was shown by Schrage ( 19681 ) that SRPT min- 
imizes the number of customers in the G/G/1 queue over all work-conse r ving d i sciplin e s. Fo r 
early references on these and other scheduling disciplines see iKleinrockl (|l97fil ): IWolfj (Il989l ) 
and the references therein. Recently, the pe rformance of these d isciplines was revisited in the 
context of heavy tails; for a recent survey see lBorst et al.l (|2003bl ). For practical applica t ions o f 
SRPT-based scheduling to impro ving Web server performance see Harchol-Balter et al. ( 20031 ): 
Rawat and Kshemkalvani ( 20031 ): a.lso, for re c ent studies that are applying FBPS to reducing 



the latency of short TCP flows see lRai et aP (l2004l . l2005l ). 

It is well known that the sojourn time distributions under PS, FBPS and SRPT scheduling 
disciplines are asymptotically equivalent for power law distributions (more prec isely, regularly 
or inte rmediately regularly varying distributions) . This was originally proved by iNuhez-Queiia 
(|200nl ) and then l a ter stu died for regularly varying distributions in Theorems 2.2, 2.5 and 
2.6 of Borst et al. ( 2003b); see also Theo r em 2 .1 of Jelenkovic and Momcilovic ( 2003bl ) and 
Theorem 1 of Jelenkovic and Momcilovic ( 20021 ). In other words, for large jobs, the waiting 
time does not depend on the choice of a specific scheduling discipline among PS, FBPS and 
SRPT. 

In this paper, we introduce a new notion of conditional waiting time distribution which 
allows us to refine and distinguish the performance of PS/FBPS and SRPT schedulers for 
medium size jobs. Informally, our first main result, stated in Theorem 11.21 shows that even 
the relatively smaller jobs receive asymptotically the same residual capacity 1 — p as the larger 
ones for SRPT discipline, while, for PS/FBPS schedulers, these smaller jobs share the residual 
capacity equally with the larger jobs in the system. Hence, it appears that SRPT provides 
much better and more uniform performance over a wide range of time scales. Furthermore, 
the performance improvement for conditionally smaller jobs is not achieved at the expense 
of larger jobs, i.e., SRPT is not only efficient but fair as well, which is in line with simila r 
recent findings in the contex t of mean value analysis by Bansal and Harchol-Balter ( 200ll ): 
Wierman and Harchol-Balteil ([^03). To this end, we wo uld like to point out that con trary 



to our findings, in the light-tailed context, it was shown by lRamanan and Stolyaii (120011 ) that 
FIFO is optimal in terms of maximizing the decay rate of the waiting time distribution over 
all work conserving discipline s. For more recent results on the light-tailed asymptotic analysis 



see 



Nuvens and Zwart ( 20061 ) and the references therein. 



Overall, using the SRPT scheduling is beneficial for a broad range of c onditions and appli- 
cation s. However, as discussed in one of the very first papers on SRPT bv lSchrage and Miller 
(|l966l ). this discipline may be quite difficult to implement. Clearly, its complicated preemp- 
tive nature requires keeping track of the remaining processing times for all jobs in the queue 
which may be prohibitive for systems with large job volumes, e.g., Web servers. In addition. 
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Schrage and Miller ( 19661 ) show that the expected number of preemptions per job is propor- 



tional to the l oad of the system, which c an be quite large. Hence, even as early as 1966, it was 
recognized by Schrage and Miller ( 19661 ) that one should try to approximate SRPT with less 
complex schedulers. The most apparent option, as suggested by ISchrage and Miller (|l966l ). IS 
to design a threshold-based static priority approximation to SRPT. Basically, the idea is to 
select a fixed number of thresholds m and then group jobs into m + 1 classes depending on 
which pair of thresholds a job size happens to fall between. Then, these classes are served 
according to the static priority discipline with higher priorities assigned to classes with smaller 
jobs. Since then, there has been a lot of work on th reshold-based scheduling policies. For 
example, it was shown by Bansal and Gamarnik ( 20061 ) that even with a single threshold, one 
can obtain the performance comparable to SRPT up to a constant factor in terms of the mean 
sojourn time for M/M/1 queue as well as for M/G/1 queue with finite variance Pareto service 
distribution. 

Although it is encouraging that one can achieve a provably very good approximation of 
M /G/l/SRPT queue even with a very small number of static thresholds (only one in the paper 
by Bansal and Gamarnik ( 20061 )). these solutions are likely not to perform well in practice since 
the traffic characteristics are often nonstat ionary, highly correl a ted (l or ig range depende i it) ari d 
very bursty (e.g., batch arrivals, etc); see iPark and Willinger (I2OOOI ): ISquillante et~ID (|l999l ). 
In order to overcome these difficulties, we propose a novel adaptive job classification (grouping) 
mechanism that is based on relative size comparison of a newly arriving job to the previous m 
arrivals; this scheduler is inspired by our conditional limit results. Specifically, if an arriving 
job is smaller than k and larger than m — k oi the previous m jobs, it is routed into class k. 
We also discuss refinements of the comparison grouping mechanism that improve the accuracy 
of the classification for both light-tailed and correlated job arrivals at the expense of a small 
(fixed) additional complexity in Subsection 12.1.11 

The good performance of our comparison classification mechanism is demonstrated using 
the asymptotic queueing analysis under heavy-tailed job sizes in Section [2^21 First, in Subsec- 
tion 12.2.11 we study the queueing behavior of a class k process in isolation and show that the 
workload distribution decays faster for larger k. More precisely, for regularly varying (power 
law) service distribution, the tail of the workload distribution P[VF*^'^^ > x] of a class k pro- 
cess, as implied by Theorem 12.11 is of the order of x(P[-B > x\)^~^^ , where B is the service 
requirement of a typical job before the comparison splitting. Hence, our comparison splitting 
procedure provides a proper ordering of jobs. Furthermore, in Subsection 12.2.2] we study the 
joint queueing behavior of all classes under the static priority (SP) discipline, with higher prior- 
ities assigned to classes with "smaller" jobs. Theorem 12.21 shows that the workload distribution 
of a class with a smaller index k (i.e., larger jobs) has the same queueing behavior as if it were 
served in isolation with the system capacity reduced by the mean arrival rates of the classes 
with smaller jobs. Roughly speaking, this is a similar behavior as seen in Theorem 11.21 for the 
SRPT discipline and, thus, the SP scheduling with our comparison splitting should provide a 
reasonable approximation to the SRPT discipline. Furthermore, in regard to the analysis, we 
would like to point out that the main technical difficulty is that the split processes are indi- 
vidually and mutually correlated. This statistical correlation makes other types of analyses, 
outside of the heavy-tailed context, possibly difficult. 

In ad dition, we would like to point out that a preliminary version of this paper has appeared 
earlier in lJelenkovic et al.l (|2007l ) as part of the conference proceedings, which contains sketches 
of the proofs as well as the extensive simulation experiments. Those experiments demonstrated 
the good performance of our adaptive scheduler that, in particular, outperforms the static 
threshold policies when the arrival processes are statistically correlated and t ir ne va rying. 
However, in contrast to the previous focus on simulations in iJelenkovic et al.l (j2007l ) . this 
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paper provides the rigorous details of the proofs. 

The rest of this paper is structured as follows. In the next section, we introduce the new 
notion of conditional waiting time distribution that refines and differentiates the performance 
of PS/FBPS and SRPT schedulers for medium size jobs. Based on the conditional asymptotic 
result of the sojourn time distribution (stated in Theorem ll.2p . we propose a novel comparison 
grouping scheme and its refined version in Section [2.11 To demonstrate its good performance, 
we conduct the asymptotic queueing analysis under heavy-tailed job sizes in Section 12.21 In 
the end. Section [3] summarizes our contributions. 



1 Heavy- Tailed Limits for Medium Size Jobs with Popular 
Schedulers 

1.1 Definitions and Prefiminary Results 

In this section we introduce the necessary notation and describe the existing and preliminary 
results. Let Bi and Vi denote the job size and the waiting time of the customer arriving at time 
Tj, respectively, where {Bi} are i.i.d. random variables. The arrival points {^}2>— oo are 
assumed to be Poisson with rate A and independent of job requirements {Bi}j>_oo- Hence, 
without loss of generality, in view of the PASTA property, we set Tq = 0. The waiting time of 
a customer is defined as the amount of time between its arrival and departure, also referred to 
as sojourn time in the queuing literature. To present our main results, we need the following 
definitions. 

Definition 1.1 A nonnegative random variable X or its distribution function (d.f.) F is called 
intermediately regularly varying, X £ ZTZ, if 

Regularly varying distributions TZa are the best-known examples from ITZ. 

Definition 1.2 A nonnegative random variable X or its d.f. F is called regularly varying with 
index a, X £ IZa {F G TZa), if 



F{x) 



1 



l{x) 



a > 0, 



where l{x): 



is slowly varying, i.e., lim^j^oo = 1, r/ > 1. 



The preceding class includes the well-known power law distributions, e.g., F{x) = 1 — 1/x", x > 
l,a > 0. 

Let Bi,l < i < m he the order statistics of B-i, 1 < i < m with the convention Bq = oo 
and Bm+i = 0. To make the notation uniform, we assume that Bq = oo, i?i = for m = 0, 
and when it is necessary to emphasize the total number of random variables, we write explicitly 



Definition 1.3 Let A^^'^ = {i?^."^ < So < -8^"^} form>k>Q. 

The asymptotic behavior of the sojourn tin ie distribution for PS, FBPS and SRPT has bee n 
extensively s tudied u nder heavy-tails, e.g., see Zwart and Boxma ( 2000 ): Nuhez-Queiial (120001) : 
Borst et al.l (|2003bl ): I.Telenkovic and MomcilovicI (j2002l ): I.Telenkovic and Momcilo^dF(|2003bl ) 



Ml 
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and the references therein. We summarize these results for intermediately regularly varying 
distributions in the following theorem, which follows directly from our more general/refined 
result presented in Theorem 11.21 in the following section. In order to ease the notation we 
simply write B = Bq and V = Vq. 

For the rest of the paper, we assume that the system has reached stationarity. Also, we 
use H to denote a sufficiently large positive constant. The value of H is generally different in 
different places, for example, H/2 = H, = H, H -\- 1 = H, etc. Furthermore, we use the 
following standard notation. For any two real functions a{t) and b{t) and fixed £ 1^ U {oo} 
we will use a{t) ~ b{t) as t ^ to to denote limt_»tQ[a(t)/5(t)] = 1. Similarly, we say that 
a{t) > h{t) a.s t ^ Iq if liminf^^tQ a(t)/b{t) > 1; a{t) < b{t) has a complementary definition. In 
addition, we say that a(t) = o{b{t)) as t ^ to if liuit^to o.{t)/b{t) = 0. When to = oo, we often 
simply write a{t) = o{b{t)) without explicitly stating t — > oo in order to simplify the notation. 

Theorem 1.1 If B £ ITZ and EB" < oo for some a> 1, then, under the PS, FBPS or SRPT 

discipline, we have, as x ^ oo, 



F[V > x]r^F[B > {I- p)x] . 

The preceding asymptotic insensitivity of the sojourn (waiting) time d istribution on the sch edul- 
ing discipline was first derived in Theorems 5.2.3, 5.2.4 and 5.2.5 of iNuhez-Queii a un- 
der som ewhat more restrictive conditions; see also Theor ems 2.2, 2.5 and 2.6 of Borst et alj 
(l2003bl ^. ?OT PS, this result was proved in Theorem 2.1 of iJelenkovic and Momcilovic (,2003bl ) 
using a novel sample path approach that allows further extension of the result to moderatel y 
heavy distributions, e.g., lognormal, see Theorem 3.1 of Jelenkovic and Momcilovic ( 2003bl ). 
Furthermore, as noted in Appendix B of I Jelenkovic and Momcilovid (|2002l ). this sample path 
approach extends directly to SRPT and FBPS scheduling di sciplines. Our proof of Theo r emll.2l 
in this paper relies directly on the arguments developed by Jelenkovic and Momcilovic ( 20021 ): 
Jelenkovic and Momcilovic ( 2003bl ) . 



1.2 Conditional Limits 

The following theorem represents our first main result, which implies Theorem 11.11 by uncon- 



ditioning on event i.e., summing over all A;, < A; < m. 



Theorem 1.2 If B £ ITZ and KB" < oo for some a > 1, then, under either PS or FBPS 
discipline, we have for fixed k, as x ^ oo. 



V>x,At^ 



B>^l^,At^' 



[1 + k) 



and under the SRPT discipline. 



1 



k + l\k 



m 



B > 



k + 1 



k+l 



:i.i) 



V>x,A^^^ 



i?> (i-p)x,4"^ 



1 



k + l\k 



m 



'[B > (1 -p)x 



fc+i 



(1.2) 



Remark 1 These results can be easily extended to GI/GI/l queue unde r the FBPS discipline , 
and possibly under the SRPT as well using the recent studies on SRPT bv lNuvens et~al] (|2007l ). 
In order to provide a unified framework, we omit such possible extensions here and restrict 
ourselves to the M/G/1 framework. Furthermore, our focus in the second part of the paper is 
to exploit this idea of relative job comparisons to design adaptive and efficient approximation 
of SRPT, which we term comparison scheduling. 
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Remark 2 Note that on Aj^ , the distribution of B has a much Hghter tail of the order of 

¥[B > x]'^^^ and, thus, A^^^ partitions the probabihty space into jobs of decreasing sizes as 
k increases. Interestingly, the result shows that, for the SRPT discipline, even the relatively 
much smaller job receives the entire long-term residual capacity 1 — p, while, for PS/FBPS, 
this smaller job shares equally the residual capacity with the k larger ones. Hence, SRPT 
outperforms PS/FBPS for medium size jobs and therefore provides much better and more 
uniform performance over a wide range of time scales, i.e., it appears that SRPT generates 
extra capacity. Informally, we believe that the explanation for this comes from the combined 
effect of the SRPT prioritization mechanism and the fact that jobs of "different" sizes occur 
on different time scales. Hence, the medium size jobs are basically not affected by the larger 
ones because of the higher priority assigned to them and the larger jobs are not impacted by 
the smaller ones due to the time scale separation. 

In order to prove this theorem, we define the class of heavy-tailed distributions C that contains 
sub exponential distributions and, in particular, the intermediately regularly varying class ITZ, 
and establish the following two preliminary lemmas. 

Definition 1.4 A nonnegative random variable X or its d.f. F is called heavy-tailed X £ C 
(or F £ C) if, for any fixed y G M, 

¥[X>x-y] 
hm — —7 — = 1. 

x^oo F{X > y] 

Lemma 1.1 Let {Xjjo^j^m be i.i.d. random variables with Xq S C and, denote the order 
statistics of Xi, X2, • • • , X,^ by Xi ^ X2 ^ • • • ^ X^ with Xq = 00 and X^+i = 0, then, for 
any m > k > 0, as x ^ 00, we have 

T[Xa > I. -t < X„ < Xt] ~ P[X„ > I, X„ < 

Remark 3 This result holds for all continuous distributions without the assumption Xq £ C. 
However, the assumption Xq G C is necessary in general since the result may not hold for 
light-tailed lattice valued distributions. Here, easy calculations show that the lemma does not 
hold for geometric distribution ¥[Xi = j] = p^{l — p),j > 0, where we obtain for m = k = 1 
and positive integer x S N 

FlXi > Xo>x]=E \l{Xo > x}p^"+^] = -^mXo >x]f. 

1 +p 

Lemma 1.2 // two arrival processes {(Tj, i?ij)}j>-oo cmd {{Ti, B2i)}i>-oo, satisfying Bu = 
for i < 0, BiQ = B20, and either Bu = B2i or Bu = for i > 0, are served with SRPT 
discipline, then, the corresponding sojourn times Vi and V2 for the customer arriving at Tq 
satisfy V2 > Vi. 

The proofs of Lemma ll. II and 11.21 are presented in the Appendix. 

Proof of Theorem II. 2t Label the customer that arrives at time Tq, and define function 
Ro{t) = RBoit) for t > to be the amount of remaining work of the labeled customer at time 
t. Let Lm be the number of customers in the system just before time T_m. For all the customers 
arriving between T_m and Tq, define B^- to be the remaining service time of -B_j, 1 < i < m 

at time t = 0. For all the customers arriving before time r_m, define B^^\m),l < i < Lm 
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to be the remaining service time at time T_m and B^^\o) the remaining service time at time 
t = 0. Denote x Ay = min(x, y). 

1. Processor sharing discipline. Similarly as in Jelenkovic and Momcilovic ( 2003bl ) . we 
have the following min-plus identity 



^0 = ^0 + 5^ 5^ A Bo + 5^ Bf^ (0) ^B^+Y,B^^ Ro{Ti). 



N{Vo) 



:i.4) 



i=l 



1=1 



i=l 



First, we establish an upper bound for (jl.ip . Observing that the residual service for 
customer —i at time is upper bounded by its original job size and using B^^\{)) < B^^\m) 
as well as Ro{Ti) < Bq, we derive on set 



(m) 



Vo <Bo + Y,B-iA Bo + )(m) A + A Bq 

i=l i=l 1=1 

Lm N{Vo) 

<{k + l)Bo + (m - k)Bl^\ + B'f\m) A Bq + ^ B, A Bo, 



1=1 



i=l 



where B, 



(m) 
m+1 



for m = k. Then, for < 6 < 1 — p, we have 



Voil-p-d)>x,A): 



(m) 



< 



ik + l)Bo > {l-6)x,Ar'\ + 



+ . 



P+S 



5x 



BABo ^ 3 



(m) 



,i=l 



Ii{x) + hix) + hix) + h{x) 



[1.5) 



where W^'^^^^ denotes the stationary workload in a queue with job sizes {Bi A Bo}i>i and 
service capacity p + 8. Now, Lemma 1 1 . 1 1 implies 



h{x) = ¥ {k + l)Bo > (1 - 6)x,A 



k 



1 



k + l\k 



m 



Bo > 



(1 - 6)x 
k+1 



k+l 



[1.6) 



Then, denote the order statistics of {B^i}^^-^^^ by {-Bj^"^^^^}o^i<m- For k = m, we have 

1, we obtain, fron 

6x 



l2{x) = 0. And, for ^ A; ^ m — 1, we obtain, from Lemma 1 1.1 1 and Bo S ITZ 

5x 



h{x) 



{m-k)B]:i[>-,Al 



5(m+l) 



3(m — k) 



m+1 
k + 2 



Bo > 



k+2 



o{h{x)). 



1.7) 



3{m — k) 

Following the same technique that was developed by Jelenkovic and Momcilovic ( 2003bl ) 
we have 



h{x) = F 



< 



p+S Sx (m) 



Bo>6^x,A^^^ 



7 



which, by Lemma 3.2 (i) in lJelenkovic and Momcilovid (|2003bl ). imphes that for 6 smah enough, 

lk+l\ 



hix) = o 



B > 



k + l 



o{h{x)). 



(1.8) 



Again, similarly as in Jelenkovic and Momcilovic ( 2003bl ). for any integer uq, we have 



hix) 



.i=l 



n=l 



k 

Y,Bl'\m)ABo>^,A 



5x . (m) 

k 



1=1 



no 



^ noP 

= hi + /42 • 

Here, it is easy to see that 

hi ^ nl 
= o(¥ 

Furthermore, since 



Y^Bl^\m)ABo>^-^,A^r^ 

i=l 



+ £ (1 - 



5x ^(^) 

3n 



^0 > — >A 



(1.9) 



(1.10) 



sup — 

xe[o,oo) p 



Bo>x,A^^^ 



Bn > 2x,A 



(m) 



< oo, 



we obtain that, for any e > 0, n > 1, there exists > such that 



3n 



Bo > —,Af^ 



< 5riog2(")ip 



3 ■ 



^0 > — ,A 



<i^,(l + e)"I 



(5x 

5n > —,A 



,(m) 



which, by choosing e small enough with r/ = p(l + e) < 1, yields 

(^a; (m) 



E (1 - ^)^' 



n=no 



5n > —,A 



3n 



n=nQ 



Sx 

Bo > —,A 



,{m) 



,"0 



^0 > Y'-^fc 



1 — 7/ 

By combining (jl.9p . (jl.lOp and (ll.llh . and then passing tiq ^ oo, we obtain 

1 k+l\ 



(1.11) 



h{x) = o 



B > 



k + l 



o(/i(x)). 



which, in conjunction with (jl.Sp . ()1.6p . (jl.7p . (jl.Sp . and by passing 5 — > 0, yields 



Vo>x,A'' 



(m) 
k 



< 



A; + 1 



(1.12) 



Next, we prove a lower bound for (jl.ip . Observe that within ■A^i^\ we have 

m N(Vo) N{V„) 

Vo^Bo + Y,B'L,^Bo+ B,A Ro{Ti) > {k + l)Bo + mT.^ + ^ 5, A Ro{Ti), (1.13) 

i=l i=l i=l 

where in the last inequahty we apphed {x — y) A z > x A z — y ior any x,y, z ^ 0; recall that 
T-rn < 0. Then, using the same arguments as in equation (3.11) in the proof of Theorem 2.1 
Jelenkovic and Momcilovic ( 2003bl ) . and the properties of for Bq E TTZ, we have 



m 



Vo{l- p)>x,At^ 



> 



(1.14) 



Combining (|1.12p and (|1.14p completes the proof of (jl.ip for PS. 

2. FBPS discipline. The proof is based on the sojourn time identity for FBPS 

N{Vo) 



Vo = Bo + WBAB^{n) + BiA Bo, 



i=l 

where WBABoiTn) denotes the stationary workload at T„ in a queue with Poisson arrival job 
sizes equal to {Bi A -Bo}-oo<i<ra and capacity 1; recall that Tq = 0. 

First, we establish an upper bound. Observe that within the set -4.^™\ 

N{Vo) 



Vo<{k + l)Bo + (m - A:)^g + WbaBo (T^m) + ^ B, A Bq, 
which, for < 5 < 1 — /), implies 



Vo(l-/5-5)>x,4'") 



< 



(A; + 1)^0 > (l-(5)x,^ 



(m) 
k 



+ : 



WB^BAT-„.)>^-^,At^ 



+ \ 



{m-k)Bl^[ >—,Al 



W 



p+5 ^ Am) 

baBq ^ 3 ' 



^/l(x)+/2(x)+/3(x)+/4(x). (1.15) 

Using the same arguments as in the proof of the upper bound for the PS case, we obtain 



h(x) 



1 f m 



k + l\k 



Bo> 



il-6)x 
k + 1 



k+l 



(1.16) 



and similarly as in (jl.7p . (jl.Sp . it follows that h^x) = o(/i(x)), l3(x) = o{Ii{x)), Ii{x) 
o(/i(x)). Therefore, by (jl.l5p and (|1.16l) . we have 



For a lower bound, within A^u"\ we obtain 



< 



(1.17) 



NiVo) 



Vo>{k + l)Bo + T_^+ Y B,A Bo, 



1=1 



9 



which is further lower bounded by the righthand side of (jl.lSp . Combining (jl.l4p and (jl.l7p 
completes the proof of (II. ip for FBPS. 

3. SRPT discipline. A similar sojourn time identity as in (|1.4p can be derived for SRPT, 



Vo = Bo + Y,Bf\Q)l{Bf{^)<B,] + Y,B''_MB\<B^]+ ^ Ba{B,<R,{m, 



N{Vo) 



i=l 



i=l 



i=l 



where we use the convention that the jobs with earlier arrivals are served first in the case of 
equal remaining service times. 

First, we prove a lower hound. For / > 0, define Bn = for i < 0, and Biq = Bq, 
Bii = Bil(Bi ^ I) for z > 0. For the new queueing system with the arrival process {{Ti,Bii)}, 
denote by {Wi(t)}t^o the workload in the system without the labeled customer. Now, define 
the stopping time Tiq = mi{t : Ro{t) ^ /} and the corresponding residual capacity without the 
labeled customer C{t) = l{Wi{s) = 0)ds. Clearly, 



E[C{t)] {I - pi)t ast^oo. 



(1.18) 



where pi = XE[B1{B < I)] = limt-,oo'^[Wi{t) > 0]. When Bq > /, ah the arrivals after time 
Tq = have shorter job requirements than the remaining service time of the labeled customer 
before time Tiq, and thus, the labeled customer can only receive service when there are no 
other customers present in the queue except itself. Therefore, conditional on {Bq > I}, we 
have 

C{Tiq) = Bq-1. (1.19) 
Next, by the standard queueing stability result and (jl.lSp . we have, for e > 0, 

Z = sup (C(t) -{1- pi + e)t) < oo. 
t>o 

From (I1.19p . Viq ^ Tiq and the monotonicity of C{t), we obtain, conditional on {Bq > /}, 

Z ^ C{Viq) -il-pi + e)ViQ ^Bq-1-{1-pi + e)ViQ, 
which, for large x, implies 

^f\Bo>1,Bq-1- Z > {1- pi + e)x,Al 
Bq -I > {l + e){l- pi + e)x,A[ 



ViQ>x,At^ 



[Z>e{l-pi + e)x\. (1.20) 



Furthermore, since the service requirements are bounded by /, the b usy period distri- 

bution of the corresponding wo r kload Wi{t) is exponentially bounded (e.g., see lNuvens and Zwart 
(|200()l ): IKhnowski and Rolskil (j200(il )^. implying that there exists 5 > 0, such that P[Z > x] = 
0{e~^^). This bound and (jl.20p . combined with Lemma 11.21 and B G ZTZ, yield 



lim ■ 

a;— >oo ■ 



BQ>{l-p)x,At^ 



^ lim 



p 


Bq > (l + e)(l 


-pi + e)x,At\ 


p 


So > (1 - 


p)x,At\ 





Passing I — > oo, e — > in the preceding inequality, we obtain the lower bound for SRPT. 

For an upper bound, since the number of customers in system for SRPT at any time is 
not larger than the number of customers in system for any other rul e applied on the same 
sequence of arrivals and service requirements, as shown by lSchrage we use the stationary 
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(PS) 

number of customers Lm at time T^m in the corresponding PS queue to upper bound L^. 

Furthermore, the workload W observed at time T^m is an upper bound for the residual work 
Ri of a customer at time T-m- Therefore, 



r(PS) 



N{V„) 



Vo<Bo + ^WABo + Y,B-il{B_i ^Bo- T.^} + ^ Bi A Bq, 



i=l i=l 

which, for any < 6 < 1 — p, yields 



Vo{l- p-S) >x,A^ 



(m) 
k 



Bo > 



APS) 



6X_ Jra) 

3m' *^ 



+ : 



Am) 



i=l 



3m 



+ ] 



BABo ^ 3 ' ^ 



^/i(x)+/2(.t)+/3(x)+J4(x). 

Similarly as in the proof of the upper bound for PS, we have 



h{x) 



1 (m 



k + l\k 



■[Bo>{l-6)x]) 



k+l 



(1.21) 



(1.22) 



and 



hix) = o(/i(x)), hix) = o{h{x)). 



(1.23) 

The only difference, as compared to the PS case, is to evaluate l2{x). Noting that A^^^ is 
a subset of 

{Bq ^ Sfe} = {-S-ii ^ Bq, • • • , B_i^ ^ Bq}, 

we obtain 



m 



Sx 

B-i > - — , Bq ^ Bk, B-i < Bq 
6m 



= Pl+P2 



Sx 

Bq+ I T-rn 1^ -B-l > - — , -Bo ^ Bh, B-i ^ Bq 
6m 

(1.24) 



where Pi is derived by upper bounding the indicator function in l2{x) by 1. To estimate Pi, 
we use 



Pi ^ 



m — 1 
k 

m — 1 
k 



5_i > So > P_i, fl {P-i ^ Po} 

k+2 

= o(/,(x)). 



6m 



(1.25) 



Next, for y = 5x/{3m), it is easy to see 



P9 ^ 



m — 1 
k-1 



Po+ I 1^ 5-1 > y, n i^-i ^ 



l<i^k 
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where the preceding probability is further bounded by 

P >y,Bo^B^i^Bo + ^F[Bo;?y- ^]''-' + P[| |> ^] 
^ (P[5_i ^ y, B-i ^Bo + ^]- P[S_i ^ y, B_, < Bq]) 

X P [So ^ y - Vy]'''^ + me-^v^/™. (1.26) 

Since Bo, B_i G J7^ and P[S_i ^ y, ^ Sq + v^] < P [^o ^ V? ~ P[5-i 5* y, ^-i < Sq], 
the right-hand side of inequahty (|1.26p is asymptotically equal to 



o(p[i?o^y]'+') =o(/i(x)), 



which, in conjunction with ()1.25p and ()1.24p . implies l2{x) = o(Ii(x)). Finally, by replacing 
(jl.22p . (jl.23p and the preceding estimation of 12(2;) in ()1.2ip . and then passing 5 ^ 0, we finish 
the proof. □ 



2 Adaptive and Scalable Comparison Scheduling 

Motivated by our conditional limits presented in Section [H we propose a novel adaptive and 
scalable comparison scheduling scheme. 



2.1 Comparison Splitting 

In this section, we describe a new adaptive job classification mechanism that we term compar- 
ison splitting. The classification is based on relative size comparison of the arriving job to the 
previous m arrivals, m > 1. Specifically, if an arriving job is smaller than k and larger than 
m — k of the previous m jobs, it is routed into class k, ^ k ^ m. 

More formally, upon the arrival of job i > 0, we define Bn > Bi2 > • • • -Bim to be the 



Bi-i] with BiQ = 00 and B. 



i(m+l) 



0. Then, if 



order statistics of {Bi^m, Bi-m+i, ■ 

Bi(^k+i) — Bi < Bif^, the new arrival Bi is routed to class k,0 < k < m and the ith arrival in 

(k) 

class k is denoted as B^ . In order to initiate the comparison splitting process, assume that 
Bi,—m ^ i ^ —1 are already known; otherwise, one can simply set Bi = 0,— i ^ —1. 

Here, we exemplify our splitting mechanism for m = 3 by dividing jobs into four classes S 
(small), M (medium), L (large) and XL (extra large) with the following rule. 



B^ £ < 



S if Bi < Bi3, , 

M if Bi3 < Bi < Bi2, 

L if Bi2 < Bi < Bii , 

XL if Bii<Bi; 



this example is depicted in Figure [H (A) . 

Now, we argue that our comparison splitting actually does order jobs into classes that 
contain smaller jobs for larger class indexes. Indeed, when B £ C, Lemma 1 1 . 1 1 yields 



Sf ^ > X 



1 



k + 1 



F[B > X 



k+l 



3jS X 



00, 



(2.1) 



which implies a decreasing distribution tail when k increases. Since the preceding expression 
is only an asymptotic result, it does not provide information on the possible ordering of the 



distributions P 



Bf^ > X 



for finite x. We address this question in the following example. 
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(A) Comparison splitter. 



Service Requirement: t 

(B) Job size distributions. 



Figure 1: A comparison splitter with m = 3 and job size distributions for four different classes. 



Example 1 In this example we simulate the performance of the comparison splitter for m = 3 
(4 classes). Assume that the job sizes are distributed as power law F(x) = 1 — 1/x" with 



a = 1. 44, which is the empirically measured file distribution by iJelenkovic and Momcilovic 



see Figure 1 on p. 577 therein. For a sample of 10^ trials, we plot the simulated 



distributions of jobs for each class in Figured] (B). From the figure, it can be observed that 



the distributions P 
asymptotic ones. 



^ > X 



are properly ordered for all values of x and k, not only for the 



Based on the previous analysis and simulation example, we can see that our comparison 
splitter has the following advantages: 

• it is adaptive since the comparing thresholds are defined by the preceding m arrivals; 

• it is scalable because the system only needs to know the sizes of the previous m jobs; 

• it provides accurate job classification as shown by equation ()2.ip and Figure [1] (B). 
Although our comparison splitter is very likely to provide a satisfactory ordering of distri- 



butions 



fif ^ > X 



, it may make errors on a sample path basis. Namely, it can occasionally 
classify smaller jobs into classes of smaller indexes and vice versa, and thus, possibly give a 
less accurate classification than a splitting mechanism that uses fixed thresholds. However, 
this possible small loss of accuracy is a fundamental tradeoff to gain the adaptability that is 
highly desirable in practice. 



2.1.1 Refined Splitting 

From the description of the comparison splitter, we can see that its adaptive thresholds are 
determined by the order statistics of the previous m arrivals. Thus, it is reasonable to expect 
that, at least for a stationary input, the accuracy of the classification will increase if we obtain 
these thresholds using a longer history (than the preceding m arrivals). However, the increase 
of history may reduce the adaptability and add to the complexity of the algorithm. 

Here, we describe one such improved comparison splitter that is based on the order statistics 
of the preceding ml, I > 1 arrivals and parameterized by {m,l). Among other reasons, we 
continue to use the order statistics since the ordered list is easy to maintain dynamically. The 
splitter works as follows. At the time of arrival of a new job i, the algorithm maintains the job 



13 



sizes of the previous ml arrivals, and orders them as Bn > B-^ > • • • > B(j_ .^iy, when needed, we 
use the notation = Bij for improved clarity. We pick the subsequence Bf^^ji^ 1 ^ j ^ m 

as the thresholds with -B(i^o) = B{i,{m+i)i) = 0) then, the new arrival is grouped into 
class k if its size lies in [B^^^^^i-^, 1 ^ ^ ^ + 1. 

In terms of engineering applications, this refined splitting algorithm is appealing because 
it can improve the accuracy for other types of arrivals, such as dependent processes and con- 
centrated discrete distributions of job sizes. In order to measure how well the refined splitter 
classifies the input sequence, we compare the output of the refined splitter with a perfectly 
ordered input sequence. Denote the input sequence by {-Bjji^i^n, the output of the refined 
splitter by {Oi}i^i^„, and the increasing order of {Bi} by {S'jji^j^n. The output of the re- 
fined splitter {Oi\ is obtained by concatenating sequentially class j + 1 after class j for all 
1 ^ j ^ m — 1. Now, define the error rate to be 

1 " 

ry(n)^-^l{O,/50. 



Lemma 2.1 For any fixed < e < 1, fixed m large enough, and an i.i.d. input sequence 
{Bi}i^i^n taking finite number of values P[-Bi = hj\ = pj,l ^ j ^ v with the splitter initialized 
by ml i.i.d. random copies of Bi that are independent of {Bi}i<^i^n, there exists i?e,^e > 0, 
such that 

P[r?(n) > e] ^ ^^g-C. min(/,n)_ 
The proof of Lemma 12.11 is presented in the appendix. 



2.2 Queueing Analysis 

In this section, we study the queueing performance of our comparison based scheduler assuming 
that jobs arrive according to a stationary renewal process {Tn}, r_i < < Tq with finite mean 

E[T] < oo, where T = Ti — Tq. The job sizes {Bn} before the splitting are i.i.d and independent 
of {Tn}- To simplify the notation and analysis in this section, we say that the ith arrival to 
class k is equal to B^^'^ = Bil{Bi(^j^j^i-^ ^ Bi < Bik}- This notation takes into account all the 
original arrival points even if i?jl{i?j(;j+i) < Bi < Bik} = 0. The addition of zero size jobs in 
each class has no impact on queueing, but simplifies the exposition. 

In Theorem 12.11 we characterize the workload asymptotics when each class is served in 
isolation. Then, in Theorem 12.21 we study the workload asymptotics of each individual class 
assuming that all the classes are served jointly according to a static priority discipline. 



2.2.1 Queueing in Isolation 



We first study the queueing characteristics of each class k when it is served in isolation with 
capacity Ck, < k < m. We use W^^^ to denote the stationary workload of class k and define 

Theorem 2.1 If¥[B > x] = e 7^„, a > 1 and E[B(^)] < Ck^[T], then, as x ^ oo, 

1 



> X 



/ p 




' X 





du 



1 fm\ l{xf+^ 

{k + l)(cfcE[r] - E[SW]) \ k) ■ 
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Remark 4 Note that this theorem indicates that the workload distribution decays faster for 
larger k. To be more specific, the tail of the workload distribution for class k decays as 
x{¥[B > x])'^"'"^ and, thus, the jobs will have the waiting time distribution of the same order 
if served under FIFO. If, for example, each class were served according to PS/FBPS, one can 
expect that the waiting times will be of the same order as {F[B > x])^~^^, as in our Theorem 
1.2i However, this is much more difficult to prove because of the dependency in {B^''}. 



Remark 5 Note that the result of Theorem 12.11 is of the same form as the one derived by 
Pakej (jl975l ) for the GI/GI/1 queue. However, Pakes's result does not apply directly to 



our 



case smce is m-depende n t. For generalizations o f Pakes's result to dependent processes 



see 



Jelenkovic and L aza P ("19981): [Xlmussen et all (|l999l l. Note that, in principle, the approach 



from lAsmussen et al. (1999.) can be applied to prove our theorem. Instead, we present a direct 



proof that may be of independent interest. 

In order to prove this theorem, we need the following definitions and lemmas. Define the 
partial sum of a stationary process {Xn}nen, where X„ G TZa, as follows, 5*0 = 0, 

n 

Sn = ^X„ n>l. (2.2) 

i=l 



Definition 2.1 For a stationary process {Xn}n£n oind m £ N, we say the process is in- 
dependent if Xn is independent of {Xi}i^n-m for all n. 



Lemma 2.2 If we define 



then 



Proof: Define 





1=0 




sup5„ > X 


^ mP 


supSi'") > - 


n>0 




.n>0 m 




L-J 

'-in-' 





q{m,j) _ \ " Y, . 
1=0 

where 1 ^ j ^ m, and observe that Sn < X^jLi •S'n"'"''* • Therefore, 



sup Sn > X 

n>0 



supX]4"'^) 



> X 



n>0 



J2 



> X 



sup^i™'^) > 

n>0 



X 

m 



n>0 



sup 5^"") > 
,n>0 



m 



where the last equality follows from the stationarity of {Xn}- 



□ 



Lemma 2.3 For a stationary m-dependent process {^n}neN with mean KXi = —5 < and 
Xi £ TZa, we have 



sup Sn > X 

n^Hx 



1 



^ [ I F[X^ > u]du 
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Proof: For simplicity of notation, in this section, we assume that Hx G N. Then, we define 
M = sup„^o with E[X„] = —6, and note that 

sup Sn = Shx + sup {Sn - Shx)- 
n^Hx n^Hx 

Since the process is stationary, we obtain 

sup {Sn - Shx) = M, 

n^Hx 



and therefore, P [sup^^j;^^, Sn > x] is upper bounded by 



5Hx ri X SHx 

^Hx H 7: h sup [Sn - Shx) 7Z- > 

^ n>Hx ^ 



36 Hx SHx 
Shx H -, — > — -, — 







+ p 


2 J 





h + h. 



From the result of Faked ( 1975 ) and Lemma 12.2^ recalling that Xi € T^q., we have 



o 



[Xi > u]du . 



(2.3) 



Similarly, by defining X^ = Xn + (35) / 4 with the partial sum Sn = Yli -^i noting that 
Sfj^ < sup„>o •S'n) 'we obtain 



h ^ 



sup Sn > 



5Hx 



(2.4) 



Combining (|2.3p and (j2.4p completes the proof. □ 
Proof of Theorem I2.lt B y the classical result of Lovned ( 19621 ) (see also Chapter 2.2 of 



Baccelli and Bremaud ( 19941 )). we have 



1 ' 



where W^'^^T^i) is the stationary workload observed at the moment T_i. Furthermore, 

Vr('=)(T_i) = sup„>o5n, with Sn = EtlX^,n > l,5o = and X^ ^ sf^ - Ck{Ti - T^^i). 
Next, observe that for x > 



[Vl/W(T_i) >x]=F 


sup Sn > X 


< F 


sup Sn > X 


+ P 


sup Sn > X 




n>l 




ni^Hx 




n^Hx 



< 



sup 5^ > 6x 

n>l 

h{x) + hix) + hix) 



sup 5„ > (1 — 6)x 

l<n<Hx 







+ p 


sup Sn > X 




n^Hx 



(2.5) 



where X^ = Xil{Xi > ex}, XI = Xil{Xi ^ ex}, and S^ = E7=i {X" + E[Xi] + 5) , S^ 
Er=i (Ki - - 6) are defined for some e > 0, \E[Xi]\ > 6 > 0. 



Firs t, let us prove an upper bound for (|2.5|) . By Lemma 3.2(i) in lJelenkovic and Momcilovic 
(|2003bl l. for any /? > 0, there exists e > such that 



/i(x) = o{x-'^). 



(2.6) 
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Furthermore, define Nk = -'-{"^1 > OjjO < k < m; note that depends on the class 



index k since Xi = — Ck{Ti — Tj_i). To simphfy the notation, we assume that Hx is an 
integer. Now, F[Ni: ^ 2] is upper bounded by 

B^''^ > ex] f""^ ^^P[5 > ex] + (^^^^ (w [fiW > ex])^ = o (P[B > x])* 



In the preceding expression, the first term bounds the sum of probabilities FlX^ > 0, X ,■ > 0] 
for all indices 1 < \i — j\ < m (note that in this case Xl and X] are dependent); the second 
term provides a bound on the corresponding sum when |i — j| > ni, using the fact that Xl and 
X'j are independent. Therefore, 



h(x) € 



sup 5„ > (1 — 6)x, Nk = 1 

0<n<Hx 



+ > 2] 

< ^ P [xl + n(E[Xi] + (5) > (1 - 6)x] + o (^x {F[B > x])^'+^ 

n=l 

^ F[Xi > (1 - (5)x + 'u|E[Xi] - 6\]du + o (^x (F[B > x 



/fx 



fe+1 



(2.7) 



The estimate for Isix) follows from Lemma 12.31 Using this estimate, ()2.5p . (j2.6p . (j2.7p and 
passing 5, e — > 0, if — > oo, we obtain the upper bound. 
Next, we prove the lower bound for ()2.5p 



»[T^W(T_i) > x] > 



sup Sri> X 
l<n<Hx 



> 



sup > X 

l<n<Hx 

Hx 



> 



sup > X, Nk = 1 

l<n<Hx 



V P [X- + n(EXi + 5) > x] > / P [Xl > X + u\EX 
n=i -^1 



which by passing x — > oo, using regular variation, and then passing 5 — > 0, results in 

1 



p[w(^)(r_i) >x]> 



/ P 


B^^'^ > u] 


1 X 





du. 



(2.J 



Finally, for any < e < 1, we have 



> X 



+ . 



> ex 



which, by (j2.8p . and then passing e ^ 0, yields 



> X 



< 



VFW(T_i) > X 



(2S 



Also, since W^^'^'^(T_i) is heavy-tailed and independent of T_i, we obtain 



> X 



W^^\t^i) + CfcT_i > xl ~ P \w^''\t^i) > X 



(2.10) 
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Thus, (EJ]) and (f230|) imply 



> X 



VF('=)(r_i) > 



(2.11) 



which, combined with (j2.8p . completes the proof of the first asymptotics. The second asymp- 
totic relationship of the theorem is implied by Lemma ll.ll □ 



2.2.2 Static Priority 



In this subsection, we assume that there is only one server with capacity c and that the m + 1 
classes are served jointly with a preemptive static priority (SP) discipline between classes. 
Suppose that the priorities of the classes are assigned in a decreasing order of the class index 
k, ^ k ^ m, i.e., class k receives service only if classes i,k + 1 ^ i ^ m are empty. Denote by 
Wq''^ the stationary workload of class k observed at arrival point Tq. Let fi^''^ = Yl^k"^ [B^^^~\ 
and note that =E[B]. 



Theorem 2.2 IfF[B > x] = l{x)/x"' G 7^Q, a > 1 and E[B] < cE[T], then, as x 

Wi'^ > x' ^ 



oo, 



poo 

/ p 


> U 


du 


1 X 







cE[T] - fiW 

(A; + l)(cE[r] -/xW) U, 



„ak+a—l ' 



(k) 

Remark 6 This result shows that the distribution of the workload Wq behaves asymptot- 
ically as if class k were served in isolation by a system with capacity reduced by the mean 
job sizes of classes with indices greater than k, which indicates a similar phenomenon as in 
Theorem 12.11 Thus, our SP scheduling with comparison splitter should approximate SRPT 
well. 

Proof: Let W^''^ (T„) be the stationary workload of class k jobs at time r„. First, we establish 
an upper bound. For ^ /c ^ m, we group all the arrivals of classes k, - ■ ■ ,m into a new class 
with the highest priority, while all the other classes remain the same. The workload of the new 
class is denoted as W^''\Tn), where W^''\Tn) = YT=k W^^i^n) and W^^'^ represents a variable 
that is equal in distribution to W^^\Tn). Clearly, 



where the workload recursion for the new class satisfies 

/ m 



(2.12) 



T^('^(T„+i) 



n+1 



c(r, 



n+l 



i=k 




Now, by Lemma [TTH it is easy to see that, as x — > oo. 



> X 



.i=k 



and, using the same argument as in the proof of the upper bound in Theorem 12. H we obtain 



(k) 



> X 



1 

cE[T] - n 



{k) 



poo 

/ p 


B^^^ > U 


1 X 





du, 



(2.13) 
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which, by (pl^ . yields 



< 



1 



cE[T] - fiC") 

Next, we prove a lower bound. For e > and k < m, we have 



> u 



du. 



(2.14) 



Wf^ > ^ ex 



Wl^''^ > (1 + e)x,W'(j^'+^^ ^ ex 



W^''^ > (1 + e)x 



Wq ' > ex 



Using the same argument as for (j2.13p and passing e ^ in the preceding inequaUty imply 

1 



W^'"'^ > X 



> 



cE[T] - fi 



(k) 



poo 

/ F 




1 X 





du. 



The same asymptotic inequality can be easily shown to hold for k = m. This inequality, 
combined with ()2.14p . completes the proof of the first asymptotic relationship in Theorem 12.21 
The second asymptotics follows directly from Lemma ll.ll □ 



3 Conclusion 



We show in Theorem 11.21 that the medium size heavy-tailed jobs can have asymptotically much 
shorter sojourn times under SRPT than under PS/FBPS scheduling disciplines. Furthermore, 
the asymptotic performance of SRPT is uniformly good for the smaller as well as for the 
larger jobs, which implies that the performance gains of smaller jobs with SRPT, compared 
to PS/FBPS, are not achieved at the expense of larger jobs. Hence, in this asymptotic heavy- 
tailed context, SRPT is both efficient and fair, which complements similar findings obtained 
using the mean value analysis. 

However, as early as in the paper by Schrage and Miller ( 19661 ) . it was observed that SRPT 
may be difficult to implement because of its complicated preemptive nature that requires 
keeping track of the remaining processing times for all the jobs in the queue. Thus, it is 
natural to consider thr eshold-based static priorit y (SP) disciplines to approximate SRPT, as 
suggested originally by Schrage and Miller ( 19661 ). which was then followed by a considerable 
number of later studies. However, the main drawback of selecting static thresholds in practice 
is that the real world traffic is often nonstationary, highly correlated, bursty, etc. 

Our second main contribution in this paper is the design of a scalable (low complexity) 
and adaptive comparison scheduling approximation to SRPT. The good performance of our 
comparison scheduler is demonstrated using our asymptotic queueing analysis under the heavy- 
tailed service re q uiren ients; additional verification of this scheduling algorithm was done by 
Jelenkovic et al. ( 200?! ) via simulations. We also discuss refinements of our mechanism that, 
at the expense of a small additional complexity, improve the accuracy of job classification for 
correlated arrivals and highly concentrated service distributions. 

Finally, we would like to point out that, in addition to the static priority discipline analyzed 
in our paper, it may also be interesting to analyze the perform ance of our splitting mechanism 
for other discip lines, such as g enera lized processor sharing in iBorst et al.l (j2003al ) , weighted 
fair queueing in ICaorita et all (|2006l l. and hierarchical processor sharing. 
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Appendix 



Proof of Lemma 11.11 

Since the case m = is immediate, we assume that m > 1. First, we show that the second 
asymptotics in ()1.3p holds assuming that {Xi}o<i<m are continuous. In this case, we have 
F[Xi = Xj] =0,iy^j and, thus 



¥[Xo>x,Xo<Xk 



m 
k 

1 / m 



X(f > X, Xq < min Xi 

l<i<k 







min Xi > X 


= ' 1 






0<i<k 



k + l\k 



F[Xo > X 



fe+i 



Next, the first asymptotics in (|1.3p is impUed by the preceding analysis and the following 
identity 

F[Xo > X, Xk+i <Xo< Xk] = nXo >x,Xo< Xk] - ff^[^o > x, Xq < Xk+i]- 

If {Xi}o<i<m are not continuous but in £, p.3|) still holds asymptotically. This claim will 
follow from the preceding arguments if we show that for Xi £ C, as x — > oo. 



P[X„ > Xn-l • • • > Xo > X] ~ F[Xn > Xn-1 ■ ■ ■ > Xq > x]. 



(3.1) 



Since Xi G C, it is enough to prove the preceding relationship for x E N. Our proof starts with 
n = 1, 



F[Xo >x,Xo< Xi] = ¥[Xo >x,Xo< Xi] + F[Xo >x,Xo = Xi]. 
Furthermore, for any e > and x large, 

oo 

W[Xo > x,Xo = Xi] =Y,ny < Xo < y + l,Xo = Xi,y < Xi < y + 1] 

y=x 

oo 

Y,{¥[y<Xo<y + l]f 



(3.2) 



< 



y=x 

oo 



Y^F[y < Xo < y + l]^J^^^F[Xo > y + 1] 



[^0 > y + 1] 

- ^P[y < Xo < y + l]P[Xo > y + 1] 

y=x 

oo 

< e ^F[y < Xo < y + l]P[Xo > y + 1] 



y=x 
oo 



(3.3) 



y=x 



^ e(P[Xo > x]f 



where the last inequality is implied by the monotonicity of P[Xo > y] and (|3.3p follows from 
Xq G C since for any e > 0, we can choose xq such that for y > a: > xq. 



iP[^o > y] 
P[Xo > y + 1] 



< l + e. 
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Combining (j3.2p . (j3.3p . using the fact that P[Xo > x,Xo < Xi] is of the same order as 
(P[Xo > a;])^, and passing e ^ 0, yield the proof for n = 1. Now, for n ^ 2, we have 

¥[Xn ^ Xn-l • • • ^ Xo > X] = P[X„ > X„_i ^ • • • ^ Xo > X] + P[X„ = ^ • • • ^ > x] 

^ P[X„ > ^ • • • ^ Xo > x] + P[X„ = > x]P[X„_2 ^ • • • ^ Xo > x] 

= P[X„ > X„_i ^ . . . ^ Xo > x] + o (P[Xo > x]"+i) , 

and by repeating the preceding procedure n — 1 more times, we obtain 

P[X„ ^ X„_i • • • ^ Xo > x] = P[X„ > Xn-i > • • • > Xo > x] + o (P[Xo > x]"+i) . 

Noting that P[X„ > X„_i > • • • > Xq > x] is of the same order as P[Xo > x]^^^ and 
P[X„ ^ X„__i • • • ^ Xo > x] ^ P[X„ > Xn-i > • • • > Xo > x], we finish the proof. □ 

Proof of Lemma 11.21 

Let Rio{t) and i?2o(^) be the remaining service times at time t ^ for the labeled customer 
that arrives at Tq under processes {(Tj, i3ij)}j>_oo and {(Tj, i?2j)}i>-oo, respectively. By the 
same notion, we define Wi{t) and W2{t) to be the workloads at time t in these two queues 
that need to be finished before the labeled customer can start receiving its service. In order to 
justify Vi ^ V2, it is enough to prove that Rio{t) ^ R2o{t),t ^ 0. 

We use induction to prove the result and denote max(x, 0) by x"*". First, if i?io(Ti) < i?2o(^i) 
and Wi{Ti+) (^W2{Ti+), we have 

Wi{t) = {Wi{T,+) -{t- T,))+ < {W2m+) -{t- T,))+ = W2{t) 
Rioit) = RioiTi) -{t-T,- Wi{Ti+))+ < R2o{Ti) -{t-T,- W2{Ti+))+ = i?2o(t) (3.4) 

for Ti < t < Tj+i. Note that Wj{Ti+) and Wj(Ti—) denote the right- and left-hand limits of 
Wj(t) at Ti, respectively; i.e., the times right after and before the arrival at T,. Hence, it is 
enough to prove that, all the customers arriving at Ti, Tq < Ti < Vi, see i?io(Tj) ^ R2o{Ti) 
and Wi{Ti+) ^ W2{Ti-\-) immediately after their arrival. 

For the arrival at time To, the claim is obviously correct. Now, assuming that the result 
holds for i = n, we proceed to prove it for i = n + 1. Based on the hypothesis, ()3.4p implies 
Rio(Tn+i) ^ i22o(^n+i) and Wi(Tn+i — ) ^ W2(Tn+i—) at the time immediately before Tn+i- 
Next, at time Tn+i, if Bi^^^+i) = < i?2(n+i)i then, we have 

Wi{Tn+l+) = Wi{Tn+l-) < W2{Tn+l-) + 52(n+l)l {52(n+l) < R2o{Tn+l)} = W2iTn+l+), 

since Wi{Tn+i-) ^ W^2(r„+i-). 

The case -Bi(„+i) = B2(^n+i) = ^n+i results in the following three different scenarios: 

1) If Bn+i < Rio{Tn+i), then 

WiiTn+l + ) = WiiTn+1-) + Bn+1 ^ Ty2(T„+l-) + B^+l = W2{Tn+l + ), 

since Rio{Tn+i) ^ R2o{Tn+i) by induction hypothesis. 

2) IfBn+i >i?2o(r„+i), then 

Wi{Tn+l+) = Wi{Tn+l-) ^ W2{Tn+l-) = W2{Tn+l+). 
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3) If Rw{Tn+i) ^ Bn+i ^ R2o{Tn+i), then 

Wi{Tn+l+) = VFl(T„+i-) ^ W2{Tn+l-) + Bn+1 = I^2(T„+l + ). 



Therefore, the result holds for i = n + 1, which completes the induction, and implies that 
V2 > Vi. □ 



Proof of Lemma 12.11 

Without loss of generality we assume that hi > ■ ■ ■ > by and Ta.\n{pk}i^i^y > 0. Define 
Qk = X^i^iPi) I ^ k ^ u with qo = and choose m > mm{l/pk}i^k^u- When Bi = bk, we 
say Bi is routed into the right class if Bi is either in class [mqk-i\ or in class \mqk~i — 1] 
(note that if mq^-i ^ N, then \jnqk-i\ = \mqk-i — 1]). The condition m > min{l/pfc}i^fc^j^ 
guarantees that if Bi 7^ Bj, then, the corresponding right classes for Bi and Bj are different 
since mpk > 1 for all 1 ^ A; ^ j^. 

First, since both {Oi} and {Si} are random, we construct a deterministic sequence {(ijji^i^n 
for comparison purposes as follows: di = bk, [nq^^il +1 ^ i ^ [nq^l- Then, 



¥[r]{n) > e] ^ P 



Y,HOi^d,}>^n 



.i=l 



Y,HSi^d.^>'-n 



.1=1 



= h+h. 

For /2, applying the union bound, we can easily prove that, for some if, ^ > 0, 



(3.5) 



(3.6) 



Therefore, we only need to prove that h ^ iie-^ where H,i may be different from the 
ones chosen in (|3.6p . 

Next, in order to evaluate Ii, we denote the event £i = {Bi is not in the right class} and 
prove that there exists if, ^ > 0, such that as n — > 00, 



max P[^:,] ^ He'^K 



(3.7) 



To this end, if = 1, it is obvious that P[<fj] = for all i]\iu'^ 2, noting that P[<Si, Bi = bi] = 0, 
we have 



^[8i] = Y,n£uB, = bk+i] 



(3.8) 



k=l 



where Pffi,^^ 



5, 



is upper bounded by 



c 



\bk+i 



< B 



(j,[mqfc+lJO 



+ 



bk+1 ^ -B(j,|-mgj^_l]/) 



ml 



l{Bi < bk+i} > [mqk + l\l 



4 = 1 



+ 



ml 



1{B, < ^ \mqk - l]l 



.i=l 



(3.9) 



By noting that E[l{Bi < bk+i}] = qkA ^ k ^ v — 1, and using the large deviation results with 
the condition [rnqk + Ij > mqk > \mqk — 1] , we obtain that for all 1 ^ A; ^ — 1 and some 
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H,^ > 0, the righthand side of (j3.9p is further bounded by He By substituting this upper 
bound for (13. 9p into (13. Sh . we prove (|3.7p . and therefore, the total number of jobs 



i=l 



that are not in the right classes satisfies, for < 6 < 1 and some Hs, ^ > 0, 



P[iVe > H = P 



^ 1{£,} > 5n 



.1=1 



5n 



(3.10) 



Now, we continue with evaluating Ii. Since 



A;=l 



,i=l 



en 
2^ 



(3.11) 



we only need to show that for each \ ^ v and some H^,(^^ > 0, 



en 
2^ 



To this end, we define eI^^^ = l{Oi 7^ di,Oi = bk} and denote by A^fc,l ^ k ^ u 

the total number of jobs of size 6^ and by NJ^ the total number of jobs of size that are 
routed into the right class with Nq = Nq = 0. Obviously, by the definition of N^, we have 



^ for 1 ^ k ^ u. Now, we claim that, for 1 ^ k ^ 



Ei'^ =Y,H0^ ^ d^,Oi = bk} ^ 



i=l 



fc-1 

J^iV;- Ln<7fc-i + lJ 

j=0 



+ 



Y.NJ-[nqk\ 
j=0 



+ 2N,. (3.12) 



In order to prove (j3.12p . we define TZk C {1, 2, • • • , n} to be the set of all the indices of 
the jobs in {Oiji^i^i/ that are routed to the right classes for job size bk- Now, if there is no 
element i of TZk such that Oi = bk, then the total number of jobs of size bk in {Oiji^j^i/ is 
bounded by since none of the jobs of size bk are in the right classes. Thus, in this case we 
obtain 



Ei'^ ^Y.^{0, = bk}^N,. 



i=l 

Next, if TZk contains at least one index i such that Oi = bk, we can always define = min{i G 
TZk ■ Oi = bk} and cr^ = max{i G TZk ■ Oi = bk}- Then, let ^ = {i G AA | ^ i ^ cr^} and 
B = {i £ M \ [nqk-i + Ij ^ « ^ L^-'ZfcJ}- 1^ is easy to see that all the indices in A but not in B 
are contributing to e!^^^ since di ^ bk for i G ^\^, and therefore, 

Ei^^ ^\A\B\+N„ 

where contains all the errors l{Oj / di,Oi = bk} for i ^ TZk- Here, "\" represents set 
difference operation and | • | denotes the cardinality of a set. To compute the cardinality of 
the preceding set difference, we have the following four different scenarios. 
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if Tk ^ [nqk-i + Ij and ak ^ [nqk\ , then | A\B \ is upper bounded by [nqk-i + Ij — t^, 



which, by noting that Yl^=o < '^k, results in 



k-l 



ViV[-Ln<?,-i + lJ 



j=0 



if Tk ^ L^-^fc-i + IJ ^iid '^fc ^ L"''/A;J 5 then | A\B \ is upper bounded by ak — [nqk\. By 
noting that J2^=o + -^e ^ '^fc ) we obtain 

k 

i=o 



if [ngfc_i + Ij ^ Tk ^ (Tk ^ [nqkl, then, | A\B |= and E'n'^'' is upper bounded by the 
total number of jobs that are not in the right classes N^; 

if Tfc < [nqk-i + l\ ^ [nqk\ < (Tk, then, we obtain | |^ [nqk-i + l\ -Tk + ak - [nqk\, 
E^n^ ^ [mk-1 + l\-Tk + ak- [nqk\ + N, 



which, by noting that Uk ^ Z]i=o ^'j + -^e X^*^=o ^'j < '^k-, yields 



fc-i 

Y^N]-[nqk-i + l\ 
j=0 



+ ^NJ +N,-lnqk\+N, 
j=0 



Therefore, by the above arguments, we prove the claim in ()3.12p . 
Next, using (j3.12p . for 1 ^ A; ^ z^, we derive 



Y,HOi^di,o, = bk} > ^ 



,i=l 



k-l 



Y^Nj-lnqk-i + l\ 

3=0 



i=o 



+ 



+ 



NJ - [nqk 

j=0 
k 

Y^N,-lnqk\ 

j=0 



+ 2N, > 



en 
2^ 



+ 4iV, > 



en 
2^ 



k-l 



j=0 



> 



en 



en 



2Au 



+ : 



^ Nj - [nqk\ 
j=0 



> 



en 



which, by noting that E[A'j] = npj for 1 ^ j ^ u and using Chernoff bound, (|3.1ip and (|3.10p . 
implies that h ^ i^g-^ for some H,^ > 0. Combining this bound, ()3.5p and ()3.6p . we 
complete the proof. □ 
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